Papers with multilingual tasks
Theoretical Linguistics Rivals Embeddings in Language Clustering for Multilingual Named Entity Recognition (2023.acl-srw)
Copied to clipboard
| Challenge: | Existing studies have used descriptive typological features and a coarse language family classification as baselines for language clustering. |
| Approach: | They propose two types of language groupings based on morpho-syntactic features in a nominal domain and one based upon a head parameter. |
| Outcome: | The proposed methods outperform state-of-the-art embedding-based models in multilingual named entity recognition (NER) . their results suggest that theoretical linguistics plays a significant role in multi-lingual learning tasks. |
Lexicon-Enhanced Self-Supervised Training for Multilingual Dense Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent multilingual pre-trained models perform poorly on multilingual retrieval tasks due to lack of multilingual training data. |
| Approach: | They propose to mine and generate self-supervised training data based on large-scale unlabeled corpus and introduce query generator to generate more queries in target languages for unlabed passages. |
| Outcome: | The proposed method performs better than baselines on a Mr. TYDI dataset and an industrial dataset from a commercial search engine. |
Multilingual Retrieval-Augmented Generation for Knowledge-Intensive Question Answering Task (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies focus on English as the data language for RAG, resulting in limited coverage of multilingual RAG. |
| Approach: | They propose a method that translates retrieved documents into a common language before generating the response. |
| Outcome: | The proposed approach improves efficiency on knowledge-intensive tasks but introduces inconsistencies due to cross-lingual variations in the retrieved content. |
Translate-Train Embracing Translationese Artifacts (2022.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to train multilingual tasks are based on translationese and translatetrain. |
| Approach: | They propose to use translationese to mitigate the gap between the source and target languages to train the translator. |
| Outcome: | The proposed method outperforms baselines on the multilingual QA dataset TyDiQA. |
Breaking the Language Barrier: Can Direct Inference Outperform Pre-Translation in Multilingual LLM Applications? (2024.naacl-short)
Copied to clipboard
Yotam Intrator, Matan Halfon, Roman Goldenberg, Reut Tsarfaty, Matan Eyal, Ehud Rivlin, Yossi Matias, Natalia Aizenberg
| Challenge: | Existing studies have focused on pre-translation, but there is still need for it . authors say that it is not universally necessary to translate large language models . |
| Approach: | They re-evaluate the need for pre-translation in the context of PaLM2 models . authors found that PaLM2-L consistently outperforms pre-translated in 94 out of 108 languages . |
| Outcome: | The proposed model outperforms pre-translation in 94 out of 108 languages and 6 benchmarks . authors argue that pre-translated inputs can be used to improve performance . |
MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation (2025.emnlp-main)
Copied to clipboard
Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, Irene Li
| Challenge: | Existing large language model evaluation benchmarks focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities. |
| Approach: | They propose a comprehensive benchmark covering 29 languages, built on an English benchmark. |
| Outcome: | The MMLU-ProX is a comprehensive benchmark covering 29 languages, built on an English benchmark. |
Transcending Scaling Laws with 0.1% Extra Compute (2023.emnlp-main)
Copied to clipboard
Yi Tay, Jason Wei, Hyung Chung, Vinh Tran, David So, Siamak Shakeri, Xavier Garcia, Steven Zheng, Jinfeng Rao, Aakanksha Chowdhery, Denny Zhou, Donald Metzler, Slav Petrov, Neil Houlsby, Quoc Le, Mostafa Dehghani
| Challenge: | Existing scaling of language models is expensive and requires significant computational costs. |
| Approach: | They propose a method that substantially improves existing language models and their scaling curves with a relatively tiny amount of extra compute. |
| Outcome: | The proposed method significantly improves existing language models and their scaling curves with a relatively tiny amount of extra compute. |
Cross-Cultural Similarity Features for Cross-Lingual Transfer Learning of Pragmatically Motivated Tasks (2021.eacl-main)
Copied to clipboard
| Challenge: | a large amount of work on cross-lingual transfer learning focused on typological and genealogical similarities between languages. |
| Approach: | They propose three features that capture cross-cultural similarities that manifest in linguistic patterns and quantify distinct aspects of language pragmatics. |
| Outcome: | The proposed features capture cross-cultural similarities manifest in linguistic patterns and quantify aspects of language pragmatics. |
MTLS: Making Texts into Linguistic Symbols (2024.emnlp-main)
Copied to clipboard
| Challenge: | In linguistics, all languages can be considered as symbolic systems . most work overlooks the properties of languages as symbol systems - aaron et al., 1989). |
| Approach: | They propose a method to make texts into linguistic symbols to improve multilingual capability . they use a pre-training method to replace pre-trained language models with a vocabulary map . |
| Outcome: | The proposed method improves multilingual capabilities on multilingual tasks using BERT and RoBERTa as the backbone. |
Structure-Level Knowledge Distillation For Multilingual Sequence Labeling (2020.acl-main)
Copied to clipboard
| Challenge: | Existing multilingual models still underperform individual monolingual models due to model capacity limitations. |
| Approach: | They propose to distill the structural knowledge of several monolingual models (teachers) to the unified multilingual model (student). |
| Outcome: | The proposed model outperforms strong baseline models and teacher models on 4 multilingual tasks with 25 datasets and has stronger zero-shot generalizability. |
GLUECoS: An Evaluation Benchmark for Code-Switched NLP (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies show multilingual contextual embedding models perform better on cross-lingual and multilingual tasks. |
| Approach: | They propose to evaluate multilingual contextual embedding models on multilingual data . they use language identification from text, POS tagging, Named Entity Recognition and Question Answering . |
| Outcome: | The proposed benchmark evaluates models on language identification from text, POS tagging, Named Entity Recognition, Question Answering and a new task for code-switching, Natural Language Inference. |
Can we teach language models to gloss endangered languages? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior research has explored statistical and neural methods for automatically producing IGT. |
| Approach: | They propose to use in-context learning to generate interlinear glossed text . they propose to employ supervised learning to select examples to provide in-text . |
| Outcome: | The proposed methods beat standard transformer baselines, despite requiring no training at all. |
Synergy with Translation Artifacts for Training and Inference in Multilingual Tasks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has shown promising transferability of pre-trained multilingual language models. |
| Approach: | They propose a cross-lingual fine-tuning algorithm that uses SupCon and MixUp to combine them to improve performance. |
| Outcome: | The proposed algorithm improves cross-lingual transferability by using SupCon and MixUp. |
Text Rendering Strategies for Pixel Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to rendering text use a large set of almost-equivalent input patches, which may prove sub-optimal for downstream tasks due to redundancy in the input representations. |
| Approach: | They propose four approaches to rendering text in a PIXEL model using character bigrams and patch frequency biases. |
| Outcome: | The proposed models perform better on sentence-level tasks without compromising performance on token-level or multilingual tasks. |
Code-switched inspired losses for spoken dialog representations (2021.emnlp-main)
Copied to clipboard
| Challenge: | We introduce new pretraining losses tailored to learn generic multilingual spoken dialogue representations . goal is to expose model to code-switched language . |
| Approach: | They propose to build a pretraining corpus of multilingual conversations in five different languages from OpenSubtitles. |
| Outcome: | The proposed models perform better in monolingual and multilingual settings. |
Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) demonstrate impressive multilingual capability, but their performance varies substantially across different languages. |
| Approach: | They propose a generic template prompt that stimulates cross-lingual and logical reasoning skills to enhance task performance across languages. |
| Outcome: | The proposed method improves multilingual capability across languages and covers high-resource and low-resourced languages. |
Crosslingual Generalization through Multitask Finetuning (2023.acl-long)
Copied to clipboard
Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, Colin Raffel
| Challenge: | Multitask prompted finetuning (MTF) has been shown to help large language models generalize to new tasks in a zero-shot setting, but so far explorations of MTF have focused on English data and models. |
| Approach: | They apply multitask prompted finetuning to pretrained multilingual models and generate variants called BLOOMZ and mT0. |
| Outcome: | The proposed models can generalize to non-English languages that have never been seen before. |
Explainability and Interpretability of Multilingual Large Language Models: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature on multilingual large language models lacks transparency in their internal processes. |
| Approach: | They propose to use multilingual large language models to examine their explainability and interpretability methods. |
| Outcome: | The present study examines the explainability and interpretability of multilingual large language models. |
FFN Lens: How Transformers Divide Labor for Multilingual Tasks (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit strong performance on multilingual tasks, yet the process of constructing predictions in the target language remains under-explored. |
| Approach: | They propose a novel interpretability method focusing on the Feed-Forward Network (FFN) layers of Large Language Models. |
| Outcome: | The proposed interpretability method is based on the Feed-Forward Network (FFN) layer of Large Language Models. |